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1. What is meant by optimization? Give some examples of its applications. 

Def inition of Optimization 

• In engineering, physical, and mathematical sciences, computational optimization, 
or simply optimization, means either minimization or maximization of a certain 
objective function. 

Examples of Optimization Problems/ Applications 

• Optimizing the parameters of the ANN model (weights and biases). 

• Optimizing the parameters of the ANFIS model (premise and antecedent 
parameters). 

• Tuning the parameters of the PID controller (get the best value for Kp, Ki and Kd). 

• Getting the best Placement of Wi-Fi Access Point for Indoor Positioning system 


(IPS). 

• Approximating experimental, mathematical functions with the least possible error. 


• Optimizing the methods of Micro-array data analysis (Bioinformatics field). 


2. Explain the main differences between the following terms: 

> Global optimum and local optimum 

• Optimum can be global or local. To illustrate, the single-variable function/ (x) 
(represents the objective function to be minimized) in Fig.l has one global 
minimum and two local minimum; the global minimum is the least among all local 
minimum. 

• Note that every global optimum is a local optimum, but the reverse is not 
necessarily true. 
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> Unimodal and multimodal functions 


Unimodal functions 


multimodal functions 


A unimodal function has a single local 
optimum which is itself the global 
optimum 



A multimodal function has more than one 
local optimum and one global optimum 
which is a local optimum with the least 
objective function value among all other 
local optimum 



> Separable and non-separable functions 


Separable functions 

Non-separable functions 

If function f (a^. x z , , ac 0 ) can be 

If function f (x 1 . x, , , x D ) can be 

divided to D functions 

divided to D functions 

in the following form: 

in the following form: 

/(*i) + /(* 2 )+ /(*/>) 

/ (*i) + /( *z) + 
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> Univariate and Multivariate 


Univariate 

Multivariate 

Number D of variables Xd 
(Dimensionality) 

(D= 1) 

Number!) of variables Xd 
(Dimensionality) 

(D> 1) 


> Traditional optimization methods and population-based optimization methods 

Evolutionary optimization algorithms are population-based algorithms of 
candidate solutions, not just one solution as in traditional methods. 


> Exploration and exploitation 


Exploration 

Exploitation 

Exploration means finding new solutions 
(or points) in the search domains which 
have not been evaluated before. 

In exploration, the variation of the 
population members from one iteration to 
another is large. 

Exploitation means trying to improve the 
current found solutions by performing 
relatively small changes that lead to new 
solutions in the immediate neighborhood. 

In exploitation, the variation of the 
population members from one iteration to 
another is very small. 


3. How can we evaluate an optimization algorithm? 

• In order to evaluate the efficiency and robustness for an optimization algorithm, 

standard complex mathematical functions with different characteristics called 
benchmark functions are used to test the optimization algorithm. 

• After selecting a suitable set of benchmark functions, the algorithm is running 

over these functions for N independent of runs. Each run consists of determined 
No. of iterations. 

• The results of the test show the No. of successful runs for each function. The run 

is considered successful if the algorithm reached to the required global optimum. 


4. What are the basic elements of the optimization process? 

(1) An objective function / which is the function to be optimized (minimized or 
maximized). 

(2) The number of components or variables of the objective function that 
specifies the dimensionality of the optimization problem 

If the objective function/is expressed in the form/(x;, X2,..., xd) 
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Then x i,X2,...,xd are the independent variables and D is the number of 
variables specifies the dimensionality of the problem. 

The objective function can be written compactly as : 

F(x) , x = [ xi, X2,. . xd ]is a lxD vector 

(3) A set of constraints forced on the required solution 

Most problems constrain at least the search domains of the variables vector 
X = [X!,X 2 ,...,Xd] 


5. State whether the following sentences is true (T) or false (F) and correct the wrong 
ones: 

> Every local optimum is a global optimum. (T/F) False 

Correction: Every global optimum is a local optimum, but the reverse is not necessarily 
true. 

> The exploitation process is done before exploration process. (T/F) False 
Correction: The exploration process is done before exploitation process. 

> Large number of solutions in the populations results in decreasing the 
exploration rate. (T/F) False 

Correction: Large number of solutions in the populations results in increasing the 
exploration rate. 

> Any optimization algorithm starts with large exploration rate. (T/F) True 

> The more No. of successful runs for an optimization algorithm over a certain 
objective function, the less robustness of the optimizer. (T/F) False 

Correction: The more No. of successful runs for an optimization algorithm over a certain 
objective function, the more robustness of the optimizer. 

> The goal of the optimizer is to reach to a local optimum. (T/F) False 
Correction: The goal of the optimizer is to reach to a global optimum. 

> A 'good' optimizer should get trapped in only one local optimum. (T/F) False 
Correction: A 'good' optimizer should get trapped in only one global optimum. 

> The No. of objective functions to be optimized specifies the dimensionality of the 
optimization problem. (T/F) False 

Correction: The number of components or variables of the objective function that 
specifies the dimensionality of the optimization problem 

> The minimum value of / (x) is the same as the maximum value of -/ (x). (T/F) 
False 


Fuzzy Logic Control (FLC) 


Correction: The minimum value of/(x) is the same as the negative maximum value of - 

/(x). 


6. Write a MATLAB code to program to draw the 3-D map of the following 
benchmark functions: 


Benchmark Functions 

Search Range 

Rosenbrock 

Function 

D - 1 

AO) = ^ [100(xf - x i+1 ) 2 + (Xj - l) 2 ] 

i= 1 

[-2,048.2.048]° 

Griewank 

Function 

fM- 1+ y x * - rf cos( Xi ) 

[-600.600]° 

hix) - 1+ Z 4000 i l i=1 \ri) 

i=l 

Ras trigin 
Function 

D 

f 3 (x) = £[10 + xf - 10cos(2?rx i )] 

i= 1 

[-5.12.5.12]° 

Schwefe] 

Function 

D 

/ 4 (x) = 418.9829 D — £ x t sin(-/jxj) 

t=l 

[-500.500]° 


(Hint: Put D = 2 to draw the 3-D maps) 


• Rosenbrock Function 

» x=[-2:0.1:2]; 

» [xl , x2] = meshgrid(x); 

» z=100*((xl. A 2 - x2). A 2 +(xl - 1) A 2); 
» surfc(xl,x2,z) 

» colormap(hsv) 



-2 -2 


• Griewank Function 

» x=[-500: 1:500]; 

» [xl , x2] = meshgrid(x); 

» z = 1 + (xl. A 2)./4000 +(x2. A 2)./4000 - (cos(xl./sqrt(l))*cos(x2./sqrt(2))); 
» surfc(xl,x2,z) 

» colormap(hsv) 
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• Rastrigin Function 

» x=[-5:0.1:5]; 

» [xl , x2] = meshgrid(x); 

» z=(10 + xl. A 2 - 10*(cos(2*pi.*xl))) + (10 + x2. A 2 - 10*(cos(2*pi.*x2))); 
» surfc(xl,x2,z) 



• Schwefel Function 

» x=[-500: 1:500]; 

» [xl , x2] = meshgrid(x); 

» z = (418.9829*2) - ((xl.*sin(sqrt(abs(xl)))) + (x2.*sin(sqrt(abs(x2))))); 
» surfc(xl,x2,z) 

» colormap(hsv) 

2000 ? 1 i 
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